Testing is one of the components of the development process for software units. As part of this activity, tests determine whether software units correspond to a predetermined specification, for example. This is frequently an iterative process, wherein after each test of the relevant software unit any non-compliances that have been identified in relation to the specification can be remedied. Particularly in the case of safety-critical software products, the possibility must be considered that parts of the computer system on which the software is executed are faulty. A fault in a hardware unit ranges from sporadic malfunction to total failure. Using corresponding tests, it is possible to assess a behavior change in a software product, wherein said behavior change was caused by a fault in a hardware unit.
One approach for assessing effects of faulty hardware units on the execution behavior of a program flow is the utilization of software-based test codes. In this context, a test code is created which explicitly simulates the behavior changes when executing a program flow using a faulty hardware unit. In the case of the module test, for example, this is implemented by means of stubs. In the module test, individual modules are validated in respect of their functionality before they are integrated, in order then to perform the subsequent tests of a test hierarchy (e.g. integration test, system test). In this context, integration refers to the incorporation of individual software modules in an existing modularly structured software system, and to the integration of individual hardware modules into an existing computer system. In this context, a stub is a separate software module which replaces parts of other software modules, parts of libraries, or parts of the operating system. In order to test behavior in the event of hardware faults, the stubs are implemented in such a way that they replicate the behavior of other submodules in the event of a fault in a hardware unit.
A physical hardware unit is e.g. a fixed disk. In order to test a fixed disk fault, for example, operating system calls are replaced by calling the stubs that are responsible for reading and writing data. Instead of executing the functionality of the operating system, a stub will merely return an error code or an error report to the calling program flow. The stub merely replaces the extended functionality of the tested module here, while other parts of the software system use the correctly functioning operating system call as usual. This has the disadvantage that stubs can only test individual modules with regard to their behavior relative to faulty hardware units. The development cost of implementing the aforementioned stubs is also disadvantageous. It is also not possible to test the overall system as a whole by means of software-based test code. In particular, software-based test code cannot include the operating system of the physical computer system in the test of the hardware units.
A further known approach consists in replacing the correctly functioning hardware unit with an explicitly faulty hardware unit during the test. Real faulty hardware units which are utilized in precisely these test cases must be varied in respect of a predetermined functionality, such that the predetermined fault is simulated. This is disadvantageous in particular because the current market for hardware units is characterized by great diversity. Consequently, a hardware unit that is varied according to the predetermined fault would have to be provided for each physical hardware unit that is to be tested. Furthermore, when simulating a plurality of faulty hardware units, it would be necessary to simulate all combinations of the potentially faulty hardware units. In this context, however, the software system can only be validated in respect of precisely those faults for which the corresponding faulty hardware is available. Moreover, the validation in respect of suddenly occurring faults is not possible using this method. A fault which occurs during the execution of the program flow, or which occurs temporarily, cannot be simulated however.